Data storage systems are arrangements of hardware and software that may include multiple storage processors coupled to arrays of non-volatile storage devices. In typical operation, the storage processors service storage requests that arrive from hosts. The storage requests specify files or other data elements to be written, read, created, or deleted, for example. The storage processors each run software that manages incoming storage requests and performs various data processing tasks to organize and secure user data stored on the non-volatile storage devices.
It is common for data storage systems to include various caches and transaction logs to facilitate metadata processing. For example, a data storage system may employ a system cache for storing file system metadata to be written to or read from storage volumes, e.g., to access frequently-used metadata without having to perform a disk access each time the metadata are needed. A data storage system may also employ buffer caches for holding file system metadata in high-speed memory. Buffer caches may be backed by a transaction log. As is known, a transaction log is a non-volatile intent log for temporarily persisting pending metadata changes. A transaction log may be implemented, for example, as a circular buffer of limited size on one or more flash drives. A file system may quickly and persistently store pending metadata changes in the transaction log to keep pace with incoming IO (input/output) requests from hosts. As resources permit, and generally out of band with IO requests, the data storage system may sync the metadata changes stored in the transaction log to storage volumes, e.g., by writing the metadata changes to system cache, which may then flush the changes to underlying disk drives.
Transaction logs also help to maintain consistency in the event of a failure. For example, if a storage processor running on a data storage system experiences a failure after writing pending metadata changes to a transaction log, the file system can replay the transactions upon recovery to fix the pending changes in the file system's nonvolatile metadata structures, e.g., by writing them to system cache and flushing them to disk. Thus, no metadata changes are lost, and the file system remains consistent even after the failure.